{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "WfGpInivS0fG"
   },
   "source": [
    "<h2 align=\"center\">点击下列图标在线运行HanLP</h2>\n",
    "<div align=\"center\">\n",
    "\t<a href=\"https://colab.research.google.com/github/hankcs/HanLP/blob/doc-zh/plugins/hanlp_demo/hanlp_demo/zh/tok_restful.ipynb\" target=\"_blank\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
    "\t<a href=\"https://mybinder.org/v2/gh/hankcs/HanLP/doc-zh?filepath=plugins%2Fhanlp_demo%2Fhanlp_demo%2Fzh%2Ftok_restful.ipynb\" target=\"_blank\"><img src=\"https://mybinder.org/badge_logo.svg\" alt=\"Open In Binder\"/></a>\n",
    "</div>\n",
    "\n",
    "## 安装"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "IYwV-UkNNzFp"
   },
   "source": [
    "无论是Windows、Linux还是macOS，HanLP的安装只需一句话搞定："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "1Uf_u7ddMhUt"
   },
   "outputs": [],
   "source": [
    "pip install hanlp_restful -U"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "pp-1KqEOOJ4t"
   },
   "source": [
    "## 创建客户端"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "id": "0tmKBu7sNAXX"
   },
   "outputs": [],
   "source": [
    "from hanlp_restful import HanLPClient\n",
    "HanLP = HanLPClient('https://www.hanlp.com/api', auth=None, language='zh') # auth不填则匿名，zh中文，mul多语种"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "EmZDmLn9aGxG"
   },
   "source": [
    "#### 申请秘钥\n",
    "由于服务器算力有限，匿名用户每分钟限2次调用。如果你需要更多调用次数，[建议申请免费公益API秘钥auth](https://bbs.hanlp.com/t/hanlp2-1-restful-api/53)。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "elA_UyssOut_"
   },
   "source": [
    "## 分词\n",
    "HanLP线上模型训练自`9970`万字的大型综合语料库，覆盖新闻、社交媒体、金融、法律等多个领域，是已知范围内**全世界最大**的中文分词语料库。语料库规模决定实际效果，面向生产环境的语料库应当在千万字量级。自然语义的语言学专家一直在持续标注该语料库，与时俱进保持最先进的分词质量。\n",
    "在分词标准上，HanLP提供细粒度和粗粒度两种颗粒度，细粒度适合搜索引擎业务，粗粒度适合文本挖掘业务。\n",
    "### 细粒度分词\n",
    "默认细粒度："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[['商品', '和', '服务', '。'],\n",
       " ['阿婆主', '来到', '北京', '立方庭', '参观', '自然', '语义', '科技', '公司', '。']]"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "HanLP.tokenize('商品和服务。阿婆主来到北京立方庭参观自然语义科技公司。')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "用户也可以直接将`HanLP`当作函数调用，并且打印漂亮的分词结果："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 35
    },
    "id": "BqEmDMGGOtk3",
    "outputId": "6fbb3eac-df26-4a55-8ba9-975d6cede227"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div style=\"display: table; line-height: 128%;\"><pre style=\"display: table-cell; font-family: SFMono-Regular,Menlo,Monaco,Consolas,Liberation Mono,Courier New,monospace; white-space: nowrap;\">商品&nbsp;和&nbsp;服务&nbsp;。</pre></div><br><div style=\"display: table; line-height: 128%;\"><pre style=\"display: table-cell; font-family: SFMono-Regular,Menlo,Monaco,Consolas,Liberation Mono,Courier New,monospace; white-space: nowrap;\">阿婆主&nbsp;来到&nbsp;北京&nbsp;立方庭&nbsp;参观&nbsp;自然&nbsp;语义&nbsp;科技&nbsp;公司&nbsp;。</pre></div>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "HanLP('商品和服务。阿婆主来到北京立方庭参观自然语义科技公司。', tasks='tok').pretty_print()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "返回类型为[Document](https://hanlp.hankcs.com/docs/api/common/document.html)，是`dict`的子类，拓展了很多操作各种语言学结构的方法。\n",
    "\n",
    "两个接口都会对文本进行分句，所以返回的结果一定是句子的列表。推荐在不超过服务器允许的最大长度的前提下，尽量传入整篇文章，以提高分词速度。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "jj1Jk-2sPHYx"
   },
   "source": [
    "### 粗粒度分词\n",
    "执行粗颗粒度分词："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[['商品', '和', '服务', '。'], ['阿婆主', '来到', '北京', '立方庭', '参观', '自然语义科技公司']]"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "HanLP.tokenize('商品和服务。阿婆主来到北京立方庭参观自然语义科技公司', coarse=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "或者直接当函数调用："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 35
    },
    "id": "1goEC7znPNkI",
    "outputId": "ddf15a17-2f5d-4bc3-d145-908fb6176552"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div style=\"display: table; line-height: 128%;\"><pre style=\"display: table-cell; font-family: SFMono-Regular,Menlo,Monaco,Consolas,Liberation Mono,Courier New,monospace; white-space: nowrap;\">阿婆主&nbsp;来到&nbsp;北京&nbsp;立方庭&nbsp;参观&nbsp;自然语义科技公司&nbsp;。</pre></div>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "HanLP('阿婆主来到北京立方庭参观自然语义科技公司。', tasks='tok/coarse').pretty_print()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "wxctCigrTKu-"
   },
   "source": [
    "### 同时执行细粒度和粗粒度分词"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "Zo08uquCTFSk",
    "outputId": "bf24a01a-a09b-4b78-fdec-2bb705b4becb"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'tok/fine': [['阿婆主', '来到', '北京', '立方庭', '参观', '自然', '语义', '科技', '公司', '。']],\n",
       " 'tok/coarse': [['阿婆主', '来到', '北京', '立方庭', '参观', '自然语义科技公司', '。']]}"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "HanLP('阿婆主来到北京立方庭参观自然语义科技公司。', tasks='tok*')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`fine`为细分，`coarse`为粗分。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 多语种分词\n",
    "得益于语言无关的设计，HanLP支持包括简繁中英日俄法德在内的104种语言上的分词。这一切，只需指定`language='mul'`即可实现。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div style=\"display: table; line-height: 128%;\"><pre style=\"display: table-cell; font-family: SFMono-Regular,Menlo,Monaco,Consolas,Liberation Mono,Courier New,monospace; white-space: nowrap;\">In&nbsp;2021&nbsp;,&nbsp;HanLPv2.1&nbsp;delivers&nbsp;state-of-the-art&nbsp;multilingual&nbsp;NLP&nbsp;techniques&nbsp;to&nbsp;production&nbsp;environments&nbsp;.</pre></div><br><div style=\"display: table; line-height: 128%;\"><pre style=\"display: table-cell; font-family: SFMono-Regular,Menlo,Monaco,Consolas,Liberation Mono,Courier New,monospace; white-space: nowrap;\">2021&nbsp;年&nbsp;、&nbsp;HanLPv2.1&nbsp;は&nbsp;次&nbsp;世代&nbsp;の&nbsp;最&nbsp;先端&nbsp;多&nbsp;言語&nbsp;NLP&nbsp;技術&nbsp;を&nbsp;本番&nbsp;環境&nbsp;に&nbsp;導入&nbsp;します&nbsp;。</pre></div><br><div style=\"display: table; line-height: 128%;\"><pre style=\"display: table-cell; font-family: SFMono-Regular,Menlo,Monaco,Consolas,Liberation Mono,Courier New,monospace; white-space: nowrap;\">2021&nbsp;年&nbsp;HanLPv2.1&nbsp;为&nbsp;生产&nbsp;环境&nbsp;带来&nbsp;次世代&nbsp;最&nbsp;先进的&nbsp;多&nbsp;语种&nbsp;NLP&nbsp;技术&nbsp;。</pre></div>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "HanLP(['In 2021, HanLPv2.1 delivers state-of-the-art multilingual NLP techniques to production environments.',\n",
    "       '2021年、HanLPv2.1は次世代の最先端多言語NLP技術を本番環境に導入します。',\n",
    "       '2021年 HanLPv2.1为生产环境带来次世代最先进的多语种NLP技术。'], tasks='tok', language='mul').pretty_print()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "自然语言处理分为许多任务，分词只是最初级的一个。也许大家只听说过中文分词，但HanLP并不局限于分词。HanLP的使命是普及最前沿的自然语言处理技术到生产环境，所以在其他教程中你会见到许多更高级的NLP任务以及相应的API用法。"
   ]
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "collapsed_sections": [],
   "name": "tok_restful.ipynb",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}
